{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 使用镜像部署模型\n",
    "\n",
    "PAI支持用户使用镜像的方式部署模型，通过镜像，开发者可以自定义模型部署的环境，包括Python、使用的机器学习框架、依赖的第三方库等，能够支持用户灵活的部署需求。详细的介绍可以参考PAI帮助文档：[使用镜像部署模型](https://help.aliyun.com/zh/pai/user-guide/deploy-a-model-service-by-using-a-custom-image)。\n",
    "\n",
    "PAI Python SDK提供了便利的API，支持用户能够使用自定义镜像，或是PAI提供的预置推理，将一个本地，或是OSS上的模型快捷得部署为模型在线服务。\n",
    "\n",
    "本文档将介绍，用户如何通过PAI Python SDK通过自定义镜像的方式部署模型。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## 费用说明\n",
    "\n",
    "本示例将会使用以下云产品，并产生相应的费用账单：\n",
    "\n",
    "- PAI-EAS：部署推理服务，详细计费说明请参考[PAI-EAS计费说明](https://help.aliyun.com/zh/pai/product-overview/billing-of-eas)\n",
    "- OSS：存储模型和推理等，详细计费说明请参考[OSS计费概述](https://help.aliyun.com/zh/oss/product-overview/billing-overview)\n",
    "\n",
    "> 通过参与云产品免费试用，使用**指定资源机型**，可以免费试用PAI产品，具体请参考[PAI免费试用](https://help.aliyun.com/zh/pai/product-overview/free-quota-for-new-users)。\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## 安装和配置SDK\n",
    "\n",
    "我们需要首先安装PAI Python SDK以运行本示例。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python -m pip install --upgrade pai"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "SDK需要配置访问阿里云服务需要的AccessKey，以及当前使用的工作空间和OSS Bucket。在PAI SDK安装之后，通过在 **命令行终端** 中执行以下命令，按照引导配置密钥、工作空间等信息。\n",
    "\n",
    "\n",
    "```shell\n",
    "\n",
    "# 以下命令，请在 命令行终端 中执行.\n",
    "\n",
    "python -m pai.toolkit.config\n",
    "\n",
    "```\n",
    "\n",
    "我们可以通过以下代码验证配置是否已生效。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pai\n",
    "from pai.session import get_default_session\n",
    "\n",
    "print(pai.__version__)\n",
    "\n",
    "sess = get_default_session()\n",
    "\n",
    "# 获取配置的工作空间信息\n",
    "assert sess.workspace_name is not None\n",
    "print(sess.workspace_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 部署模型推理服务\n",
    "\n",
    "模型在线服务包含了模型的文件、模型的推理服务代码、以及推理服务运行环境。\n",
    "本示例将使用一个简单的`PyTorch`模型，通过`Flask`和`PAI`提供的`PyTorch`基础镜像，部署模型在线服务。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "下载示例使用的简单PyTorch模型。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 下载模型到本地 \"model\" 目录\n",
    "\n",
    "!mkdir -p model/\n",
    "!wget https://pai-sdk.oss-cn-shanghai.aliyuncs.com/pai/resources/toy_model.pt -P model/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 准备推理服务代码\n",
    "\n",
    "在部署模型之前，我们首先需要准备推理服务的代码，它提供HTTP接口，负责接收预测请求，使用模型进行推理，返回预测结果。\n",
    "\n",
    "当前示例我们将使用 ``Flask`` 编写一个简单的推理服务，保存为 ``infer_src/app.py`` 文件。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!mkdir -p infer_src"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%writefile infer_src/app.py\n",
    "import json\n",
    "from flask import Flask, request\n",
    "import os\n",
    "import torch\n",
    "import numpy as np\n",
    "\n",
    "app = Flask(__name__)\n",
    "model = None\n",
    "# 默认的模型文件路径\n",
    "MODEL_PATH = \"/eas/workspace/model/\"\n",
    "\n",
    "def load_model():\n",
    "    \"\"\"加载模型\"\"\"\n",
    "    global model\n",
    "    model = torch.jit.load(os.path.join(MODEL_PATH, \"toy_model.pt\"))\n",
    "    model.eval()\n",
    "\n",
    "@app.route(\"/\", methods=[\"POST\"])\n",
    "def predict():\n",
    "    data = np.asarray(json.loads(request.data)).astype(np.float32)\n",
    "    output_tensor = model(torch.from_numpy(data))\n",
    "    pred_res = output_tensor.detach().cpu().numpy()\n",
    "    return json.dumps(pred_res.tolist())\n",
    "\n",
    "if __name__ == \"__main__\":\n",
    "    load_model()\n",
    "    app.run(host=\"0.0.0.0\", port=int(os.environ.get(\"LISTENING_PORT\", 8000)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 获取PAI提供的预置推理镜像\n",
    "\n",
    "PAI提供了一系列预置的推理镜像，镜像内预置了机器学习框架、常用的第三方库、Python、NVIDIA CUDA库等。我们可以通过以下代码列出所有的预置镜像。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pai.image import list_images, ImageScope\n",
    "\n",
    "\n",
    "data = [\n",
    "    [\n",
    "        \"ImageUri\",\n",
    "        \"FrameworkName\",\n",
    "        \"FrameworkVersion\",\n",
    "        \"AcceleratorType\",\n",
    "        \"PythonVersion\",\n",
    "    ]\n",
    "]\n",
    "\n",
    "# 列出常用的PyTorch推理镜像\n",
    "for img in list_images(framework_name=\"PyTorch\", image_scope=ImageScope.INFERENCE):\n",
    "    data.append(\n",
    "        [\n",
    "            img.image_uri,\n",
    "            img.framework_name,\n",
    "            img.framework_version,\n",
    "            img.accelerator_type,\n",
    "            img.python_version,\n",
    "        ]\n",
    "    )\n",
    "\n",
    "# 列出常用的TensorFlow推理镜像\n",
    "for img in list_images(framework_name=\"TensorFlow\", image_scope=ImageScope.INFERENCE):\n",
    "    data.append(\n",
    "        [\n",
    "            img.image_uri,\n",
    "            img.framework_name,\n",
    "            img.framework_version,\n",
    "            img.accelerator_type,\n",
    "            img.python_version,\n",
    "        ]\n",
    "    )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.display import HTML, display\n",
    "\n",
    "display(\n",
    "    HTML(\n",
    "        \"<table><tr>{}</tr></table>\".format(\n",
    "            \"</tr><tr>\".join(\n",
    "                \"<td>{}</td>\".format(\"</td><td>\".join(str(_) for _ in row))\n",
    "                for row in data\n",
    "            )\n",
    "        )\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "通过SDK提供的 `pai.image.retrieve` API，可以获取指定框架版本的镜像。在当前示例中，我们将使用PAI提供的PyTorch 1.12版本的CPU推理镜像"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pai.image import retrieve, ImageScope\n",
    "\n",
    "# # 获取PyTorch 1.10 GPU推理镜像\n",
    "# print(retrieve(\n",
    "#     framework_name=\"PyTorch\",           # 框架名称\n",
    "#     framework_version=\"latest\",         # 框架版本\n",
    "#     accelerator_type=\"gpu\",             # 选择支持Nvidia CUDA GPU的镜像\n",
    "#     image_scope=ImageScope.INFERENCE,   # 镜像类型，推理镜像\n",
    "\n",
    "# # ).image_uri)\n",
    "\n",
    "# 获取最新的PyTorch CPU推理镜像\n",
    "torch_image_uri = retrieve(\n",
    "    framework_name=\"PyTorch\",  # 框架名称\n",
    "    framework_version=\"1.12\",  # 框架版本，latest表示使用PAI支持的最新版本\n",
    "    # accelerator_type=\"cpu\",           # 默认使用CPU镜像\n",
    "    image_scope=ImageScope.INFERENCE,  # 镜像类型，推理镜像\n",
    ").image_uri\n",
    "print(torch_image_uri)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 部署推理服务\n",
    "使用以上的推理服务代码，以及PyTorch推理镜像，我们将一个PyTorch模型部署为模型在线服务。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pai.model import Model, container_serving_spec\n",
    "\n",
    "\n",
    "m = Model(\n",
    "    model_data=\"./model/\",  # 模型文件，可以是一个本地文件或是OSS Bucket路径(例如 oss://<BucketName>/path/to/model )，\n",
    "    inference_spec=container_serving_spec(\n",
    "        image_uri=torch_image_uri,  # 推理服务使用的镜像\n",
    "        command=\"python app.py\",  # 模型推理服务启动命令\n",
    "        source_dir=\"./infer_src/\",  # 推理服务代码所在目录\n",
    "        requirements=[\"flask==2.0.0\", \"Werkzeug==2.3.4\"],  # 推理服务依赖的Python包\n",
    "    ),\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from pai.common.utils import random_str\n",
    "\n",
    "# 部署模型服务\n",
    "p = m.deploy(\n",
    "    service_name=f\"toy_model_{random_str(6)}\",  # 模型服务名称, 地域内唯一\n",
    "    instance_type=\"ecs.c6.large\",  # 模型服务使用的机器实例规格\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 调用推理服务\n",
    "\n",
    "部署服务后返回的`pai.predictor.Predictor`对象可以用于调用推理服务，发送预测请求。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "# 构造一个随机数组输入\n",
    "dummy_input = np.random.rand(1, 10, 10).tolist()\n",
    "print(dummy_input)\n",
    "\n",
    "result = p.raw_predict(\n",
    "    data=dummy_input,\n",
    ")\n",
    "\n",
    "# 打印推理结果\n",
    "print(result.json())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在测试完成之后，删除推理服务"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "p.delete_service()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "base",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
